The present invention relates to a process for automated assessment of the problem solving skill needed to solve the kinds problems that arise in mathematics, science, or technology. Such problems are formula based and have several steps such that the answer to at least one step can be obtained in at least one way by calculations that use calculated answers to one or more prior steps.
To manually grade such problems efficiently and in a way that fairly assigns partial credit, one looks for answers to key intermediate steps. If correct, then credit is given for all subordinate prior steps. If incorrect but reasonable, then an attempt is made to assign partial credit for its correct use in obtaining subsequent answers.
The invention in pending U.S. patent application Ser. No. 10/397,303, filed Mar. 27, 2003 discloses and claims a process whereby such fair, efficient grading of formula-based multi-step problems is performed by a computer via a web interface. This application was published on Sep. 30, 2004, as Publication No. US 2004/0191746 A1. Applicant herein is a co-inventor of the invention disclosed and claimed in the published patent application.
In the invention disclosed and claimed therein, memory is provided to store formula based multi-step problems, data to enable and facilitate assessment of the correctness of answers to the problem steps, and a computer program that uses these data to grade submitted answers. In the preferred embodiment, students using the system disclosed therein have access to the memory via a computer and a connection to a global computer network such as, for example, the INTERNET, and are thus able to access multi-step formula based problems and go through the steps necessary to solve them. Through communication via the global computer network, students submit answers to one or more selected problem steps to the computer program, which grades them and reports the results.
The earlier published application discloses a variety of grading strategies, including the prior and later answer strategies noted above. In the prior answer strategy, credit is given for prior steps having correct answers that were not submitted but needed in a correct calculation of a submitted answer that was graded as correct. In the later answer strategy, an incorrect submitted answer to a particular step is given credit if it could be obtained correctly from incorrect but reasonable submitted answers to one or more earlier steps.
The earlier published application also discloses various inventive ways to inform the student of progress made toward a complete solution after a set of submitted answers is graded. This information includes the problem steps that were graded as correct by virtue of the prior and later answer strategies as well as those for which correct answers were submitted. It also includes the specification of the minimal set of steps that are “required” in the sense that correct submitted answers to these steps will result in full credit for the problem.
The invention disclosed in the earlier published patent application permits complex problems with multi-step solutions to be graded effectively over a global computer network. However it lacks the capability to assess the skill level of the individuals that use it. Skill-level assessment that can rapidly and reliably identify students' strengths and weaknesses is critical to learning management and central to placing students in appropriate grade levels or courses.
To solve a complex problem, one must organize it into interrelated steps and then correctly carry out those steps. Methodologies for assessing problem-solving skill level therefore necessarily involve multi-step problems. Currently, problems used for such assessment are either solved by hand and graded manually or administered and graded on a computer. In the prior art in the latter case, the correct answer to any particular step is needed before answers to any subsequent step can be submitted. The requirement to submit answers to all steps in a pre-assigned sequence limits the ability to see how efficiently the student could have solved the problem without such constraints.
The present invention uses and improves the ability of the earlier invention to store heretofore-unavailable data, and uses these data to assess problem-solving skill level in a manner unknown in the prior art. Persons thus assessed are likely to be students, but may not be, and so will be referred to as users. Different users might take differing paths to solve the same problem, with some paths being more effective and indicative of higher levels of problem-solving skill based on some objective standard. Moreover, individuals using the same path might do so with different degrees of proficiency, as indicated by the number of submitted answer sets, the “required” vs. “optional” nature of the answers submitted on each set, and the number of times, if at all, that answers to the various problem steps were submitted before credit was given. Finally, as described herein, the preferred embodiment of the earlier invention enables automated error-locating assistance and answers in the form of strategy-describing formulas. Data describing the use of error-locating assistance and strategy-describing formulas is highly indicative of the skill level with which a problem was solved but is not available in the prior art.